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DETAILED ACTION 

1 . This office action is responsive to communication filed on November 6, 2008. 
Claims 11-14 and 17-23 are pending in the application and have been examined by the 
Examiner. Claims 15 and 16 have been withdrawn from consideration. 

Continued Examination Under 37 CFR 1.114 

2. A request for continued examination under 37 CFR 1.114, including the fee set 
forth in 37 CFR 1 .17(e), was filed in this application after final rejection. Since this 
application is eligible for continued examination under 37 CFR 1.114, and the fee set 
forth in 37 CFR 1 .17(e) has been timely paid, the finality of the previous Office action 
has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 
November 6, 2008 has been entered. 

Response to Arguments 

3. Applicant's arguments with respect to claims 11-14 and 17-23 have been 
considered but are moot in view of the new ground(s) of rejection. 

Claim Rejections - 35 USC § 102 

4. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 1 02 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351 (a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21(2) 
of such treaty in the English language. 
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5. Claims 1-3, 6, 17, 19 and 20 are rejected under 35 U.S.C. 102(e) as being 
anticipated by Edwards (US 7,312,766). 

Consider claim 1, Edwards teaches: 

A digital imaging system (figure 1) comprising: 

a. an image sensor (video camera, 1 1 ) and a separate display (Head 
Mounted Display, HMD, 1 , and first computer, 7), wherein a sensing device to which 
said image sensor is coupled (i.e. video camera, 1 1 ) operates independently of a 
display device to which said separate display is coupled (The display (1 , 7) is external 
from the sensing device (1 1 ), as shown in figure 1 and detailed in column 4, line 66 
through column 5, line 12); said separate display being capable of rotating through at 
least one angle that is independent from the rotation of said image sensor (The HMD is 
connected to the head of a user, and as the image sensor (1 1 ) is not located on the 
head of the user, the HMD rotates independently from the camera, column 5, lines 9- 
15.); 

b. an image sensor orientation sensor (The orientation of the camera is 
sensed and transmitted to the first computer (7) as camera position information, column 
5, lines 51-60. This is shown in figure 1 as "pointing mechanism position corresponding 
to camera video image".); a display orientation sensor (3-axis head tracker, 3, column 5, 
lines 10-20); and 

c. an image manipulator (first computer, 7, see figure 6), wherein the display 
device (1 , 7) includes the image manipulator (7), which: 
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i) receives image sensor orientation ("capture position data", "associated image 
and position data", and "transmit data to first computer", figure 6); 

ii) receives display orientation ("provide HMD data position", figure 6); and 

iii) adjusts the image orientation ("transform image based on offset between HMD 
and camera", figure 6). 

See also column 5, line 48 through column 6, line 9, column 7, lines 6-32. 

Consider claim 3, and as applied to claim 1 above, Edwards further teaches that 
the digital imaging system is chosen from the group consisting of still cameras and 
video cameras (The camera is a video camera (1 1 ), see figure 1 .). 

Consider claim 6, and as applied to claim 1 above, Edwards further teaches that 
the image manipulator comprises an image rotation system (see figures 3d and 3e, 
column 6, lines 20-23). 

Consider claim 2, Edwards teaches: 
A digital imaging system comprising: 

a. an image sensor for sensing an image subject and to capture a 
presentation of the image (video camera, 1 1 ); and a separate display device for 
displaying said presentation of said image (Head Mounted Display, HMD, 1, and first 
computer, 7), wherein a sensing device to which said image sensor is coupled (i.e. 
video camera, 1 1 ) operates independently of said separate display device (The display 
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(1, 7) is external from the sensing device (11), as shown in figure 1 and detailed in 
column 4, line 66 through column 5, line 12); said separate display being capable of 
rotating through at least one angle that is independent from the rotation of said image 
sensor (The HMD is connected to the head of a user, and as the image sensor (1 1 ) is 
not located on the head of the user, the HMD rotates independently from the camera, 
column 5, lines 9-15.); 

b. an image sensor orientation sensor for sensing changes in the orientation 
of an image with respect to a first image sensor base line orientation coordinates (The 
orientation of the camera is sensed and transmitted to the first computer (7) as camera 
position information, column 5, lines 51-60. This is shown in figure 1 as "pointing 
mechanism position corresponding to camera video image".); and a display orientation 
sensor for sensing changes in the orientation of said display device with respect to a 
second display device base line orientation coordinates (3-axis head tracker, 3, column 
5, lines 10-20. The display orientation sensor senses changes in orientation with 
respect to the head of the user. As the display is remote from the camera, and the 
orientation of each respective device is sensed, the display and the camera have 
different baseline coordinates.); and 

c. an image manipulator (first computer, 7, see figure 6), wherein the 
separate display device (1 , 7) includes the image manipulator (7), which: 

i) receives image sensor orientation from the image sensor orientation sensor 
("capture position data", "associated image and position data", and "transmit data to first 
computer", figure 6); 
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ii) receives display orientation from the display orientation sensor ("provide HMD 
data position", figure 6); and 

iii) adjusts the image orientation in relation to both the image sensor orientation 
and the display orientation to reconcile said different first and second baseline 
orientation coordinates ("transform image based on offset between HMD and camera", 
figure 6). 

See also column 5, line 48 through column 6, line 9, column 7, lines 6-32. 
Consider claim 17, Edwards teaches: 

A display (Head Mounted Display, HMD, 1, and first computer, 7) for an image 
produced by an image sensor component (video camera, 1) of a digital imaging system 
(figure 1), wherein said image sensor (1) operates independently of a display device to 
which said display is coupled (The display (1 , 7) is external from the sensing device 
(11), as shown in figure 1 and detailed in column 4, line 66 through column 5, line 12); 
and wherein said display being capable of rotating through at least one angle that is 
independent from the rotation of said image sensor (The HMD is connected to the head 
of a user, and as the image sensor (11 ) is not located on the head of the user, the HMD 
rotates independently from the camera, column 5, lines 9-15.); said display comprising 

a. a display orientation sensor (3-axis head tracker, 3, column 5, lines 10- 

20); 

b. means to received the orientation from the image sensor (The orientation 
of the camera is sensed and transmitted to the first computer (7) as camera position 
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information, column 5, lines 51-60. This is shown in figure 1 as "pointing mechanism 
position corresponding to camera video image".); 

c. an image manipulator (first computer, 7, see figure 6), which: 

i) receives image sensor orientation ("capture position data", "associated image 
and position data", and "transmit data to first computer", figure 6); 

ii) receives display orientation ("provide HMD data position", figure 6); and 

iii) adjusts the image orientation ("transform image based on offset between HMD 
and camera", figure 6). 

See also column 5, line 48 through column 6, line 9, column 7, lines 6-32. 

Consider claim 19, and as applied to claim 17 above, Edwards further teaches 
that the display is chosen from the group consisting of mobile displays (As the display is 
head mounted ("Head Mounted Display"), it is considered mobile.). 

Consider claim 20, and as applied to claim 17 above, Edwards further teaches 
that the image manipulator comprises an image rotation system (see figures 3d and 3e, 
column 6, lines 20-23). 

Claim Rejections - 35 USC § 103 

6. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
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invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

7. The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1 , 148 
USPQ 459 (1966), that are applied for establishing a background for determining 
obviousness under 35 U.S.C. 103(a) are summarized as follows: 

1 . Determining the scope and contents of the prior art. 

2. Ascertaining the differences between the prior art and the claims at issue. 

3. Resolving the level of ordinary skill in the pertinent art. 

4. Considering objective evidence present in the application indicating 
obviousness or nonobviousness. 

8. Claims 4, 7 and 8 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Edwards (US 7,312,766). 

Consider claim 4, and as applied to claim 1 above, Edwards does not explicitly 
teach that the image sensor is a charge coupled device array. 

However, Official Notice (MPEP § 2144.03) is taken that both the concepts and 
advantages of using a charge coupled device array image sensor in a digital camera 
are well known and expected in the art. It would have been obvious to a person having 
ordinary skill in the art at the time of the invention to us a charge coupled image sensor 
in the camera taught by Edwards for the benefit of enabling the recording of images with 
an excellent image quality and high dynamic range. 

Consider claim 7, Edwards teaches: 

A digital camera (video camera, 11, second computer, 15, figure 5) comprising: 
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b. an orientation sensor that detects the orientation of said digital camera 
relative to gravity (The orientation of the camera is sensed and provided to a computer 
(7, 15) as camera position information, column 5, lines 51-60, column 6, line 66 through 
column 7, line 3. Figure 3a shows a "nominal display of image", which image is clearly 
oriented relative to gravity.); and 

c. means to receive the orientation relative to gravity (3-axis head tracker, 3, 
column 5, lines 10-20) of a display device (Head Mounted Display, HMD, 1, and first 
computer, 7) capable of displaying the image produced by said digital camera (see 
figures 3a-4d), wherein said display device (1 , 7) operates independently of said digital 
camera (The display (1 , 7) is external from the sensing device (1 1 ), as shown in figure 1 
and detailed in column 4, line 66 through column 5, line 12); 

d. at least one image manipulator (second computer, 15, column 6, line 66 
through column 7, line 3.) which: 

i) receives said digital camera orientation ("capture position data", "associated 
image and position data", and "transmit data to first computer", figure 6); 

ii) receives said display device orientation ("provide HMD data position", figure 6); 

and 

iii) transforms the image produced by said camera to reconcile differences 
relative to gravity between said orientation of said charge coupled device image sensor 
and said orientation of said display device ("transform image based on offset between 
HMD and camera", figure 6. Since the baseline for the display and camera orientations 



Application/Control Number: 1 0/701 ,829 Page 1 0 

Art Unit: 2622 

is relative to gravity (figure 3a), the reconciliation between the HMD and camera is 
relative to gravity.). 

See also column 5, line 48 through column 6, line 9, column 7, lines 6-32. In 
addition, note that column 6, lines 66-67 recites that the processing is performed either 
in the first computer (7, i.e. the display computer) or the second computer (1 5, i.e. the 
camera computer). 

However, Edwards does not explicitly teach that the digital camera comprises a 
charge coupled device image sensor. 

However, Official Notice (MPEP § 2144.03) is taken that both the concepts and 
advantages of using a charge coupled device image sensor in a digital camera are well 
known and expected in the art. It would have been obvious to a person having ordinary 
skill in the art at the time of the invention to us a charge coupled image sensor in the 
camera taught by Edwards for the benefit of enabling the recording of images with an 
excellent image quality and high dynamic range. 

Consider claim 8, and as applied to claim 7 above, Edwards further teaches that 
the digital camera is chosen from the group consisting of still cameras and video 
cameras (The camera is a video camera (1 1 ), see figure 1 .). 

9. Claim 5, 9, 13(1), 13(2) and 14 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Edwards in view of Thomas (US 6,781 ,623). 
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Consider claim 5, and as applied to claim 1 above, Edwards does not explicitly 
teach that either orientation sensor is chosen from the group consisting of electronic 
gyroscopic sensors, mechanical gyroscopic sensors, and optical gyroscopic sensors. 

Thomas similarly teaches a digital imaging system (figures 5 and 6) comprising 
an image sensor ("CCD", 16, figure 6, column 4, lines 38-41) and a separate display 
(12), an image sensor orientation sensor ("sensor", 20, figure 6, column 4, lines 42-45), 
and at least one image manipulator ("Digital Signal Processor", 36, figure 6, column 5 
lines 18-24) adapted to receive image sensor orientation (See figure 6, the image 
sensor orientation is obtained by the frame memory (38), which is connected to the 
image sensor (16), and provided to the image manipulator (34), column 5, lines 15-22.), 
receive image orientation (See figure 6, the image orientation is obtained by the sensor 
(20) and provided to the image manipulator (34), column 5, lines 18-26.), and adjust the 
image orientation (The image manipulator (34) applies a rotational transform to the 
image, column 5, lines 18-26, figure 7, column 6, lines 33-46.). 

However, in addition to the teachings of Edwards, Thomas teaches that the 
image sensor orientation sensor is chosen from the group consisting of mechanical 
gyroscope sensors (Thomas teaches that a mechanical gyroscope sensor can be used 
to apply a rotational transform and maintain an orientation which has been manually 
aligned, column 7, line 35 through column 8, line 19.). 

Therefore, it would have been obvious to a person having ordinary skill in the art 
at the time of the invention to use a mechanical gyroscope sensor as taught by Thomas 
as the image orientation sensor taught by Edwards for the benefit of enabling the 
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detection of changes from a vertical alignment set manually by a professional 
cameraman (Thomas, column 8, lines 6-10). 

Consider claim 9, and as applied to claim 7 above, Edwards does not explicitly 
teach that either orientation sensor is chosen from the group consisting of electronic 
gyroscopic sensors, mechanical gyroscopic sensors, and optical gyroscopic sensors. 

Thomas similarly teaches a digital imaging system (figures 5 and 6) comprising 
an image sensor ("CCD", 16, figure 6, column 4, lines 38-41) and a separate display 
(12), an image sensor orientation sensor ("sensor", 20, figure 6, column 4, lines 42-45), 
and at least one image manipulator ("Digital Signal Processor", 36, figure 6, column 5 
lines 18-24) adapted to receive image sensor orientation (See figure 6, the image 
sensor orientation is obtained by the frame memory (38), which is connected to the 
image sensor (16), and provided to the image manipulator (34), column 5, lines 15-22.), 
receive image orientation (See figure 6, the image orientation is obtained by the sensor 
(20) and provided to the image manipulator (34), column 5, lines 18-26.), and adjust the 
image orientation (The image manipulator (34) applies a rotational transform to the 
image, column 5, lines 18-26, figure 7, column 6, lines 33-46.). 

However, in addition to the teachings of Edwards, Thomas teaches that the 
image sensor orientation sensor is chosen from the group consisting of mechanical 
gyroscope sensors (Thomas teaches that a mechanical gyroscope sensor can be used 
to apply a rotational transform and maintain an orientation which has been manually 
aligned, column 7, line 35 through column 8, line 19.). 
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Therefore, it would have been obvious to a person having ordinary skill in the art 
at the time of the invention to use a mechanical gyroscope sensor as taught by Thomas 
as the image orientation sensor taught by Edwards for the benefit of enabling the 
detection of changes from a vertical alignment set manually by a professional 
cameraman (Thomas, column 8, lines 6-10). 

Consider claim 13, and as applied to claim 1 above, Edwards does not explicitly 
teach of the type of orientation sensors used. 

Thomas similarly teaches a digital imaging system (figures 5 and 6) comprising 
an image sensor ("CCD", 16, figure 6, column 4, lines 38-41) and a separate display 
(12), an image sensor orientation sensor ("sensor", 20, figure 6, column 4, lines 42-45), 
and at least one image manipulator ("Digital Signal Processor", 36, figure 6, column 5 
lines 18-24) adapted to receive image sensor orientation (See figure 6, the image 
sensor orientation is obtained by the frame memory (38), which is connected to the 
image sensor (16), and provided to the image manipulator (34), column 5, lines 15-22.), 
receive image orientation (See figure 6, the image orientation is obtained by the sensor 
(20) and provided to the image manipulator (34), column 5, lines 18-26.), and adjust the 
image orientation (The image manipulator (34) applies a rotational transform to the 
image, column 5, lines 18-26, figure 7, column 6, lines 33-46.). 

However, in addition to the teachings of Edwards, Thomas teaches of using a 
mechanical gyroscope orientation sensor capable of sensing rotations in two 
dimensions (See column 8, lines 4-19. A mechanical gyroscope can be used to correct 
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the alignment of the video camera. The alignment is corrected in two dimensions, 
column 4, lines 42-62, figures 2-4.). 

Therefore, it would have been obvious to a person having ordinary skill in the art 
at the time of the invention to use a mechanical gyroscope sensor as taught by Thomas 
as the image orientation sensor taught by Edwards for the benefit of enabling the 
detection of changes from a vertical alignment set manually by a professional 
cameraman (Thomas, column 8, lines 6-10). 

Consider claim 13, and as applied to claim 2 above, Edwards does not explicitly 
teach of the type of orientation sensors used. 

Thomas similarly teaches a digital imaging system (figures 5 and 6) comprising 
an image sensor ("CCD", 16, figure 6, column 4, lines 38-41) and a separate display 
(12), an image sensor orientation sensor ("sensor", 20, figure 6, column 4, lines 42-45), 
and at least one image manipulator ("Digital Signal Processor", 36, figure 6, column 5 
lines 18-24) adapted to receive image sensor orientation (See figure 6, the image 
sensor orientation is obtained by the frame memory (38), which is connected to the 
image sensor (16), and provided to the image manipulator (34), column 5, lines 15-22.), 
receive image orientation (See figure 6, the image orientation is obtained by the sensor 
(20) and provided to the image manipulator (34), column 5, lines 18-26.), and adjust the 
image orientation (The image manipulator (34) applies a rotational transform to the 
image, column 5, lines 18-26, figure 7, column 6, lines 33-46.). 
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However, in addition to the teachings of Edwards, Thomas teaches of using a 
mechanical gyroscope orientation sensor capable of sensing rotations in two 
dimensions (See column 8, lines 4-19. A mechanical gyroscope can be used to correct 
the alignment of the video camera. The alignment is corrected in two dimensions, 
column 4, lines 42-62, figures 2-4.). 

Therefore, it would have been obvious to a person having ordinary skill in the art 
at the time of the invention to use a mechanical gyroscope sensor as taught by Thomas 
as the image orientation sensor taught by Edwards for the benefit of enabling the 
detection of changes from a vertical alignment set manually by a professional 
cameraman (Thomas, column 8, lines 6-10). 

Consider claim 14, and as applied to claim 7 above, Edwards does not explicitly 
teach of the type of orientation sensors used. 

Thomas similarly teaches a digital imaging system (figures 5 and 6) comprising 
an image sensor ("CCD", 16, figure 6, column 4, lines 38-41) and a separate display 
(12), an image sensor orientation sensor ("sensor", 20, figure 6, column 4, lines 42-45), 
and at least one image manipulator ("Digital Signal Processor", 36, figure 6, column 5 
lines 18-24) adapted to receive image sensor orientation (See figure 6, the image 
sensor orientation is obtained by the frame memory (38), which is connected to the 
image sensor (16), and provided to the image manipulator (34), column 5, lines 15-22.), 
receive image orientation (See figure 6, the image orientation is obtained by the sensor 
(20) and provided to the image manipulator (34), column 5, lines 18-26.), and adjust the 
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image orientation (The image manipulator (34) applies a rotational transform to the 
image, column 5, lines 18-26, figure 7, column 6, lines 33-46.). 

However, in addition to the teachings of Edwards, Thomas teaches of using a 
mechanical gyroscope orientation sensor capable of sensing rotations in two 
dimensions (See column 8, lines 4-19. A mechanical gyroscope can be used to correct 
the alignment of the video camera. The alignment is corrected in two dimensions, 
column 4, lines 42-62, figures 2-4.). 

Therefore, it would have been obvious to a person having ordinary skill in the art 
at the time of the invention to use a mechanical gyroscope sensor as taught by Thomas 
as the image orientation sensor taught by Edwards for the benefit of enabling the 
detection of changes from a vertical alignment set manually by a professional 
cameraman (Thomas, column 8, lines 6-10). 

10. Claims 11(1), 11(2), 12 and 21 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Edwards as applied to claims 1, 2, 7 and 17 above, and further in 
view of Ahisha(US 2005/0007477) and Riconda et al.(US 2002/013093). 

Consider claim 1 1 , and as applied to claim 1 above, Edwards further teaches: 
either of said orientation sensors produce three dimensional orientation 
information (pitch, yaw and roll, column 5, lines 12-14 and lines 57-60), and in which 
said three dimensional orientation information from either of said orientation sensors is 
used by said image manipulator to correct said image for perspective distortions that 
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result when the image sensor is located off of the perpendicular axis with respect to the 
plane of the image being sensed (Perspective distortion between the position of the 
HMD and the position of the camera is corrected, column 6, lines 6-23). 

However, Edwards does not explicitly teach using the three dimensional 
orientation information to correct for keystone, barrel and other distortions that result 
when the image sensor is located off of the perpendicular axis with respect to the plane 
of the image being sensed. 

Ahisha is similar to Edwards in that Ahisha teaches of a camera containing an 
image sensor (30) and an image processor (40, figure 1). 

However, in addition to the teachings of Edwards, Akisha teaches using 
coordinate information to correct for barrel and other (i.e. pincushion) distortions (see 
figure 2) that result when the image sensor is located off of the perpendicular axis with 
respect to the plane of the image being sensed (See paragraphs 0021-0029, especially 
paragraphs 0024, and 0026-0029). 

Therefore, it would have been obvious to a person having ordinary skill in the art 
at the time of the invention to use the coordinate information taught Edwards to correct 
for barrel and pincushion distortion as taught by Akisha for the benefit of improving 
perceived image quality. 

The combination of Edwards and Akisha does not explicitly teach correcting for 
keystone distortion. 

Riconda et al. similarly teach of a camera system for capturing and displaying 
images (see paragraphs 0062-0072, 0075). 
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However, in addition to the combined teachings of Edwards and Akisha, Riconda 
et al. teaches using coordinate information to correct for keystone distortions that result 
when the image sensor is located off of the perpendicular axis with respect to the plane 
of the image being sensed (paragraphs 0092-0103, figures 12A-12E, especially 
paragraph 0094). 

Therefore, it would have been obvious to a person having ordinary skill in the art 
at the time of the invention to use the coordinate information taught by the combination 
of Edwards and Akisha to correct for keystone distortion as taught by Riconda et al. for 
the benefit of improving perceived image quality. 

Consider claim 1 1 , and as applied to claim 2 above, Edwards further teaches: 
either of said orientation sensors produce three dimensional orientation 
information (pitch, yaw and roll, column 5, lines 12-14 and lines 57-60), and in which 
said three dimensional orientation information from either of said orientation sensors is 
used by said image manipulator to correct said image for perspective distortions that 
result when the image sensor is located off of the perpendicular axis with respect to the 
plane of the image being sensed (Perspective distortion between the position of the 
HMD and the position of the camera is corrected, column 6, lines 6-23). 

However, Edwards does not explicitly teach using the three dimensional 
orientation information to correct for keystone, barrel and other distortions that result 
when the image sensor is located off of the perpendicular axis with respect to the plane 
of the image being sensed. 
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Ahisha is similar to Edwards in that Ahisha teaches of a camera containing an 
image sensor (30) and an image processor (40, figure 1). 

However, in addition to the teachings of Edwards, Akisha teaches using 
coordinate information to correct for barrel and other (i.e. pincushion) distortions (see 
figure 2) that result when the image sensor is located off of the perpendicular axis with 
respect to the plane of the image being sensed (See paragraphs 0021-0029, especially 
paragraphs 0024, and 0026-0029). 

Therefore, it would have been obvious to a person having ordinary skill in the art 
at the time of the invention to use the coordinate information taught Edwards to correct 
for barrel and pincushion distortion as taught by Akisha for the benefit of improving 
perceived image quality. 

The combination of Edwards and Akisha does not explicitly teach correcting for 
keystone distortion. 

Riconda et al. similarly teach of a camera system for capturing and displaying 
images (see paragraphs 0062-0072, 0075). 

However, in addition to the combined teachings of Edwards and Akisha, Riconda 
et al. teaches using coordinate information to correct for keystone distortions that result 
when the image sensor is located off of the perpendicular axis with respect to the plane 
of the image being sensed (paragraphs 0092-0103, figures 12A-12E, especially 
paragraph 0094). 

Therefore, it would have been obvious to a person having ordinary skill in the art 
at the time of the invention to use the coordinate information taught by the combination 
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of Edwards and Akisha to correct for keystone distortion as taught by Riconda et al. for 
the benefit of improving perceived image quality. 

Consider claim 12, and as applied to claim 7 above, Edwards further teaches: 

both of said orientation sensors produce three dimensional orientation 
information (pitch, yaw and roll, column 5, lines 12-14 and lines 57-60), and in which 
said three dimensional orientation information from both of said orientation sensors is 
used by said image manipulator to correct said image for perspective distortions that 
result when the image sensor is located off of the perpendicular axis with respect to the 
plane of the image being sensed (Perspective distortion between the position of the 
HMD and the position of the camera is corrected, column 6, lines 6-23). 

However, Edwards does not explicitly teach using the three dimensional 
orientation information to correct for keystone, barrel and other distortions that result 
when the image sensor is located off of the perpendicular axis with respect to the plane 
of the image being sensed. 

Ahisha is similar to Edwards in that Ahisha teaches of a camera containing an 
image sensor (30) and an image processor (40, figure 1). 

However, in addition to the teachings of Edwards, Akisha teaches using 
coordinate information to correct for barrel and other (i.e. pincushion) distortions (see 
figure 2) that result when the image sensor is located off of the perpendicular axis with 
respect to the plane of the image being sensed (See paragraphs 0021-0029, especially 
paragraphs 0024, and 0026-0029). 
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Therefore, it would have been obvious to a person having ordinary skill in the art 
at the time of the invention to use the coordinate information taught Edwards to correct 
for barrel and pincushion distortion as taught by Akisha for the benefit of improving 
perceived image quality. 

The combination of Edwards and Akisha does not explicitly teach correcting for 
keystone distortion. 

Riconda et al. similarly teach of a camera system for capturing and displaying 
images (see paragraphs 0062-0072, 0075). 

However, in addition to the combined teachings of Edwards and Akisha, Riconda 
et al. teaches using coordinate information to correct for keystone distortions that result 
when the image sensor is located off of the perpendicular axis with respect to the plane 
of the image being sensed (paragraphs 0092-0103, figures 12A-12E, especially 
paragraph 0094). 

Therefore, it would have been obvious to a person having ordinary skill in the art 
at the time of the invention to use the coordinate information taught by the combination 
of Edwards and Akisha to correct for keystone distortion as taught by Riconda et al. for 
the benefit of improving perceived image quality. 

Consider claim 21 , and as applied to claim 1 7 above, Edwards further teaches: 
both of said orientation sensors produce three dimensional orientation 
information (pitch, yaw and roll, column 5, lines 12-14 and lines 57-60), and in which 
said three dimensional orientation information from both of said orientation sensors is 
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used by said image manipulator to correct said image for perspective distortions that 
result when the image sensor is located off of the perpendicular axis with respect to the 
plane of the image being sensed (Perspective distortion between the position of the 
HMD and the position of the camera is corrected, column 6, lines 6-23). 

However, Edwards does not explicitly teach using the three dimensional 
orientation information to correct for keystone, barrel and other distortions that result 
when the image sensor is located off of the perpendicular axis with respect to the plane 
of the image being sensed. 

Ahisha is similar to Edwards in that Ahisha teaches of a camera containing an 
image sensor (30) and an image processor (40, figure 1). 

However, in addition to the teachings of Edwards, Akisha teaches using 
coordinate information to correct for barrel and other (i.e. pincushion) distortions (see 
figure 2) that result when the image sensor is located off of the perpendicular axis with 
respect to the plane of the image being sensed (See paragraphs 0021-0029, especially 
paragraphs 0024, and 0026-0029). 

Therefore, it would have been obvious to a person having ordinary skill in the art 
at the time of the invention to use the coordinate information taught Edwards to correct 
for barrel and pincushion distortion as taught by Akisha for the benefit of improving 
perceived image quality. 

The combination of Edwards and Akisha does not explicitly teach correcting for 
keystone distortion. 
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Riconda et al. similarly teach of a camera system for capturing and displaying 
images (see paragraphs 0062-0072, 0075). 

However, in addition to the combined teachings of Edwards and Akisha, Riconda 
et al. teaches using coordinate information to correct for keystone distortions that result 
when the image sensor is located off of the perpendicular axis with respect to the plane 
of the image being sensed (paragraphs 0092-0103, figures 12A-12E, especially 
paragraph 0094). 

Therefore, it would have been obvious to a person having ordinary skill in the art 
at the time of the invention to use the coordinate information taught by the combination 
of Edwards and Akisha to correct for keystone distortion as taught by Riconda et al. for 
the benefit of improving perceived image quality. 

1 1 . Claims 10 and 18 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Edwards as applied to claims 7 and 17 above, and further in view of Harrison (US 
6,597,384). 

Consider claim 10, and as applied to claim 7 above, Edwards does not explicitly 
teach of manually entering orientation information. 

Harrison is similar Edwards in that Harrison teaches a mobile display device 
(figures 1a and 1b). 
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However, in addition to the teachings of Edwards, Harrison teaches that touch 
sensors (100, 102, 104, 106) are used to manually enter the display orientation (figure 
3). 

Therefore, it would have been obvious to a person having ordinary skill in the art 
at the time of the invention to manually enter display orientation information as taught by 
Harrison in the digital camera taught by Edwards for the benefit that the user can obtain 
the correct image orientation when the displayed image orientation is incorrect 
(Harrison, column 2, lines 38-39). 

Consider claim 18, and as applied to claim 17 above, Edwards does not explicitly 
teach of manually entering orientation information. 

Harrison is similar Edwards in that Harrison teaches a mobile display device 
(figures 1a and 1b). 

However, in addition to the teachings of Edwards, Harrison teaches that touch 
sensors (100, 102, 104, 106) are used to manually enter the display orientation (figure 
3). 

Therefore, it would have been obvious to a person having ordinary skill in the art 
at the time of the invention to manually enter display orientation information as taught by 
Harrison of the digital camera taught by Edwards for the benefit that the user can obtain 
the correct image orientation when the displayed image orientation is incorrect 
(Harrison, column 2, lines 38-39). 
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12. Claim 22 is rejected under 35 U.S.C. 
in view of Hinckley et al. (US 7,289,102). 
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103(a) as being unpatentable over Edwards 



Consider claim 22, and as applied to claim 17 above, Edwards does not explicitly 
teach the type of display orientation sensor used. 

Hinckley et al. is similar to Edwards in that Hinckley et al. teaches of a portable 
device (figure 3) having a display (304) and an orientation sensor (308, column 4, lines 
16-38). Hinckley et al. teach that the portable device receives image data from an 
outside source (column 2, lines 31-37, column 2, line 61 through column 3, line 3, 
column 9, lines 24-33). 

However, in addition to the teachings of Edwards, Hinckley et al. teaches that the 
display orientation sensor can consist of an electronic gyroscope sensor (column 5, 
lines 2-3). 

Therefore, it would have been obvious to a person having ordinary skill in the art 
at the time of the invention to have the display orientation sensor taught by Edwards 
consist of an electronic gyroscope sensor as taught by Hinckley et al. for the benefit of 
expanding the functionality of the display without requiring any user interaction 
(Hinckley et al., column 1 , lines 38-44). 

13. Claim 23 is rejected under 35 U.S.C. 103(a) as being unpatentable over Edwards 
in view of Hinckley et al. and Thomas. 
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Consider claim 23, and as applied to claim 17 above, Edwards does not explicitly 
teach the type of display orientation sensor used. 

Hinckley et al. is similar to Edwards in that Hinckley et al. teaches of a portable 
device (figure 3) having a display (304) and an orientation sensor (308, column 4, lines 
16-38). Hinckley et al. teach that the portable device receives image data from an 
outside source (column 2, lines 31-37, column 2, line 61 through column 3, line 3, 
column 9, lines 24-33). 

However, in addition to the teachings of Edwards, Hinckley et al. teaches that the 
display orientation sensor can consist of an electronic gyroscope sensor (column 5, 
lines 2-3). 

Therefore, it would have been obvious to a person having ordinary skill in the art 
at the time of the invention to have the display orientation sensor taught by Edwards 
consist of an electronic gyroscope sensor as taught by Hinckley et al. for the benefit of 
expanding the functionality of the display without requiring any user interaction 
(Hinckley et al., column 1 , lines 38-44). 

However, Hinckley does not explicitly teach that said gyroscope works in two or 
three dimensions. 

Thomas similarly teaches a digital imaging system (figures 5 and 6) comprising 
an image sensor ("CCD", 16, figure 6, column 4, lines 38-41) and a separate display 
(12), an image sensor orientation sensor ("sensor", 20, figure 6, column 4, lines 42-45), 
and at least one image manipulator ("Digital Signal Processor", 36, figure 6, column 5 
lines 18-24) adapted to receive image sensor orientation (See figure 6, the image 



Application/Control Number: 1 0/701 ,829 Page 27 

Art Unit: 2622 

sensor orientation is obtained by the frame memory (38), which is connected to the 
image sensor (16), and provided to the image manipulator (34), column 5, lines 15-22.), 
receive image orientation (See figure 6, the image orientation is obtained by the sensor 
(20) and provided to the image manipulator (34), column 5, lines 18-26.), and adjust the 
image orientation (The image manipulator (34) applies a rotational transform to the 
image, column 5, lines 18-26, figure 7, column 6, lines 33-46.). 

However, in addition to the teachings of Edwards and Hinckley et al., Thomas 
teaches that a mechanical gyroscope orientation sensor capable of sensing rotations in 
two dimensions can be used as an orientation sensor (See column 8, lines 4-19. A 
mechanical gyroscope can be used to correct the alignment of the video camera. The 
alignment is corrected in two dimensions, column 4, lines 42-62, figures 2-4). 

Therefore, it would have been obvious to a person having ordinary skill in the art 
at the time of the invention to use a mechanical gyroscope sensor as taught by Thomas 
as the orientation sensor taught by the combination of Edwards and Hinckley et al. for 
the benefit of enabling the detection of changes from a vertical alignment set manually 
by a professional cameraman (Thomas, column 8, lines 6-10). 

Conclusion 

14. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

1 5. Battles et al. (US 2005/008341 7) teaches of sensing a camera orientation, 
incorporating that orientation into captured images, transmitting the captured images 
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and orientation information to an external display, and displaying the images according 
to the orientation information (see figures 6 and 7). 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to ALBERT H. CUTLER whose telephone number is 
(571)270-1460. The examiner can normally be reached on Mon-Thu (9:00-5:00). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Sinh Tran can be reached on (571) 272-7564. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

AC 

/Sinh N Tran/ 

Supervisory Patent Examiner, Art Unit 2622 
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